The present invention is related to a method, an apparatus, and a program for evaluating the movements of a plurality of reference points within images that constitute a video. In addition, the present invention is related to a method, an apparatus, and a program for aligning the position of a target object included in a video and the position of simulation data of the object.
Recently, surgical simulations that employ three dimensional medical images are being commonly performed. A surgical simulation visualizes tissues and organs which are targets of surgery as well as structures in the peripheries thereof from medical images, and simulates techniques which are to be executed during actual surgery. For example, in a simulation of a partial excision of the liver, tissues including the liver, the portal vein, veins, arteries, the body surface, bones, and a tumor are extracted from tomographic images such as CT (Computed Tomography) images and MRI (Magnetic Resonance Imaging) images and visualized. The extracted tissues are visualized as a three dimensional image, and a simulation image as viewed from a surgical field during surgery is generated. Thereafter, the simulation image is employed to calculate the range of a tumor to be excised within the liver, a surgical plan is generated, and then surgery is performed.
Meanwhile, there is demand to refer to simulation images during surgery. For this reason, simulation images are printed and brought into operating rooms, or displayed on monitors provided in operating rooms. In such cases, physicians can view the printed or displayed simulation image while viewing the surgical portion of an actual patient and perform surgery.
However, performing surgery while alternately viewing a patient and a simulation image is extremely troublesome. For this reason, techniques, in which a portion which is the target of surgery is imaged during surgery to obtain a video constituted by a plurality of images and a simulation image is displayed overlapped on the video, has been proposed. For example Japanese Unexamined Patent Publication No. 2013-202313 discloses a method, in which optical sensors or magnetic sensors are directly mounted on a camera that images the target of surgery, a piece of surgical equipment, or an organ. The relative positional relationships among the sensors are calibrated. Thereafter, a simulation image is moved in response to changes in the position and orientation of the camera as well as movement of the organ after calibrating the sensors, to align the positions of the portion which is the target of surgery and the simulation image. The portion which is the target of surgery and the simulation image, of which the positions have been aligned, are overlapped and displayed in an overlapped manner.
In addition, a method, in which markers are provided at a portion which is a target of surgery, the positions of the markers are detected by a sensor, the positions of the portion which is the target of surgery and a simulation image are aligned and displayed on a head mount display, has been proposed (refer to Japanese Unexamined Patent Publication No. 2010-259497). Further, a method, in which markers are embedded in a portion which is a target of surgery, the markers are detected by a sensor, the positions of the portion which is the target of surgery and a simulation are aligned and displayed on a monitor, has been proposed (refer to Japanese Unexamined Patent Publication No. 2010-200894).
However, it is necessary to utilize expensive sensors in the methods disclosed in Japanese Unexamined Patent Publication Nos. 2013-202313, 2010-259497, and 2010-200894. In addition, there is a limited amount of space in operating rooms, and therefore it is difficult to set sensors within operating rooms. Further, there is a possibility that providing markers in targets of surgery will obstruct surgical operations.
Meanwhile, methods have also been proposed that align the positions of imaged videos of patients and simulation images without employing sensors and the like. For example, Japanese Unexamined Patent Publication No. 2012-205899 proposes a method that generates a model of an organ from a three dimensional image and displays the model of the organ overlapped on ultrasound images in real time, in which affine transform functions are obtained between each frame of the ultrasound image and the model of the organ, the model of the organ is transformed based on the obtained affine transform functions and overlapped onto the ultrasound images.
In order to align the positions of a video during surgery and simulation data, it is necessary for the simulation data to track the movements of a target portion included in the video. In order for the movements to be tracked, it is necessary to accurately detect the movements of the target portion included in the video. For this reason, a method, in which the probability that the movement will be a movement vector which is set in advance is calculated for each point within an image which is a target of movement detection, to generate a movement probability map, has been proposed (refer to Japanese Unexamined Patent Publication No. 2007-018269). In addition, a method for calculating the degree of reliability of movements that represents to what degree movements within images that constitute a video can be relied upon has also been proposed (refer to PCT Japanese Publication No. 2011-513868). By employing the methods disclosed in Japanese Unexamined Patent Publication No. 2007-18269 and PCT Japanese Publication No. 2011-513868, it will be possible to accurately detect movements among images that constitute videos.